$\alpha$-Congruence for Markov Processes

Alpha - congruence for Markov Processes

We prove innnite-time extensions of invariance principles for certain random walks with essentially compact state spaces. The extensions are uniform-like in time since they use the d-metric of the Bernoulli theory and imply the classical results. These are then generalized to couplings involving an isomorphism between the processes. In general a Doebin-type condition is needed to hold for the w...

Accelerated decomposition techniques for large discounted Markov decision processes

Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...

ON THE INFINITE ORDER MARKOV PROCESSES

The notion of infinite order Markov process is introduced and the Markov property of the flow of information is established.

The KTH Visit in Semi-Markov Processes

Markov Processes for Everybody

Introduction to the theory of continuous time Markov processes.